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SUMMARY 


1.  Background 

In  the  early  1970 's  various  economic  problems  resulted  in  increased 
difficulty  in  acquiring  defense  materiel.  For  secondary  items  we  ex- 
perienceu  higher  prices  and  more  difficulty  in  placing  contracts.  It 
was  also  felt  that  production  lead  times  (PLT)  were  increasing.  The 
Air  Force  responded  by  increasing  the  emphasis  given  to  recent  buys  in 
forecasting  PLT  for  future  buys.  The  Army,  in  an  emergency  measure,  shifted 
from  using  a  two-year  average  of  PLT  for  the  item  to  be  bought,  to  the  PLT 
of  the  last  representative  buy.  The  Materiel  Development  and  Readiness 
Command  (DARCOM)  sponsored  this  research  to  study  the  consequences  of  this 
change  and  to  improve  PLT  forecasting. 

2 .  Purpose  and  Objectives 

The  purpose  of  the  study  was  to  develop  improved  forecasting  methodology 
for  PLT  and  compensate  for  the  random  error  with  an  increase  in  safety  level 
investment.  The  individual  objectives  are  summarized  under  Scope  and  Methods 
below. 

3.  Scope  and  Methods 

In  scope  this  was  an  empirical  study  of  secondary  items  used  in  repair 
of  weapon  systems.  The  forecast  method  and  variance  estimate  are  to  be 
incorporated  into  the  Commodity  Command  Standard  System  (CCSS)  for  wholesale 
supply  management.  Specifically  they  are  to  be  made  part  of  the  replenishment 
computations  in  the  Variable  Safety  Level/Economic  Order  Quantity  module 
(VSL/EOQ) . 

The  methodology  consisted  of  assembling  a  data  base  of  aviation- 
component  procurement,  developing  forecast  methods,  testing  them  by  deter¬ 
ministic  (not  Monte  Carlo)  simulation  using  the  data  base,  selecting  the 
best-performing  method,  and  comparing  it  with  the  presently  implemented 
method. 

4.  Conclusions  and  Findings 

The  emergency  measure  -  changing  to  a  forecast  equal  to  the  latest 
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representative  PLT  -  was  found  to  be  ineffective  in  the  long  run. 

Several  better  forecast  methods  were  found.  The  best -p<- rformlng  method 
uses  the  present  age  of  buys  that  have  not  been  shipped  vet  in  an  average 
along  with  the  PLTs  of  buys  shipped  in  the  last  two  years.  It  also  makes 
use  of  an  average  of  the  PLTs  of  items  bought  on  the  same  contract  type 
to  fill  in  when  the  item's  own  data  is  limited  (few  buys  in  the  two-year 
base  period.) 

Implementing  this  method  should  considerably  improve  supply  performance 
and  reduce  safety  level  (estimated  TSARCOM  ASF  reduction  of  $15  million) . 
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INTRODUCTION 


The  purpose  of  this  study  was  to  develop  an  improved  method  of  fore¬ 
casting  production  lead  time  as  used  in  requirements  computations  for 
procurement  cf  weapon  system  support  items  on  a  recurring  basis,  and  to 
determine  the  statistical  variability  of  this  forecast  also  to  be  used 
in  the  computation  of  requirements. 

The  study  methodology  was  thus:  First  a  data  base  was  built.  This 
consisted  of  a  record  of  every  procurement  over  a  period  of  six  years  for 
all  secondary  items  bought  by  the  US  Army  Aviation  Systems  Readiness 
Command  (AVSCOM),  now  merged  with  the  US  Army  Troop  Support  Command 
(TROSCOM)  and  called  the  US  Army  Troop  Support  and  Aviation  Readiness 
Command  (TSARCOM) .  In  this  report,  the  source  of  the  data  will  be  referred 
to  as  AVSCOM.  Some  data  for  part  of  the  study  came  from  TROSCOM. 

The  second  phase  of  the  study  developed  and  evaluated  several  forecast 
methods.  These  methods  make  use  of  the  information  in  the  data  base  in 
the  same  way  they  would  when  installed  in  the  Commodity  Command  Standard 
System  (CCSS).  The  evaluation  consisted  of  comparing  thu  forecasts  at 
an  assumed  present  date  (which  were  made  using  data  which  was  known  prior 
to  the  present  date)  with  the  actual  PLT  (which  would  not  become  available 
until  after  the  present  date).  This  present  date  was  set  to  the  latest 
time  at  which  a  forecast  could  be  made  in  time  to  influence  the  buy  whose 
actual  PLT  was  used  for  comparison. 

There  was  also  an  attempt  to  identify  information  beyond  that  inherent 
in  the  procurement  history  of  the  items  stocked,  information  such  as  the 
economic  time  series  found  in  Business  Conditions  Digest.  However,  as 
reported  below,  the  most  promising  such  data  seemed  to  have  no  relation  to 
the  procurement  experience,  and  would  thus  be  of  no  use  in  forecasting. 

The  remainder  of  this  report  explains  how  the  PLT  forecast  is  presently 
made  and  how  it  fits  into  the  requirements  computation  process,  work  done 
by  other  organizations  in  this  area,  the  characteristics  of  the  data  base, 
the  forecast  methods  and  how  they  were  evaluated,  the  results  of  the  evalua¬ 
tion  including  the  final  ranking  of  the  methods  and  other  issues  that  were 
studied,  and  some  suggestions  for  additional  work. 
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CHAPTER  1 

HOW  PLT  FITS  INTO  CCSS 


1 . 1  Current  Forecast  Method 

A  PLT  is  recorded  for  each  contract  for  which  shipment  of  at  least 
one-third  of  the  quantity  ordered  has  been  completed.  The  sum  of  the  PLT 
of  the  latest  buy,  the  ALT,  and  30  days  (to  allow  for  delivery)  is  used  to 
update  the  PROI.T  field  in  the  NSNKDP.  This  field  is  subject  to  manual 
change  as  well  as,  since  May  1975,  to  an  optional  freeze  code;  i.e.  it  _an 
be  prevented  from  being  reduced. 

The  ALT  (but  not  the  delivery  time)  is  then  subtracted  from  the  PRQLT 
to  yield  the  final  value  for  PLT  which  is  used  in  requirements  computation. 
The  PROLT  is  recomputed  as  needed  by  adding  the  latest  ALT  to  the  latest 
PLT  (which  may  be  from  different  buys  because  the  ALT  becomes  available 
before  the  PLT) . 

1 . 2  Requirements  Computation 

This  PLT  is  used  to  compute  the  reorder  warning  point.  The  first  order 
effect  is  that  the  average  stock  level  increases  proportionally  with  the 
PROLT. 

The  second  order  effect  of  the  PROLT  is  to  slightly  increase  safety- 
level,  and  It  also  acts  as  an  upper  bound  on  the  safety  level. 

1 . 3  Variance 

The  variance  of  the  PLT  forecast  error  is  not  presently  used  in  re¬ 
quirements  computation,  but  methodology  for  using  it  has  been  developed 
and  It  is  now  being  incorporated  into  CCSS. 

When  a  good  PLT  forecasting  method  has  been  implemented,  the  effect 
will  be  a  reduction  in  this  variance.  Then  a  more  accurate  safety  level 
can  be  computed  by  including  this  variance.  This  will  cause  a  slight  in¬ 
crease  in  requirements  levels. 

The  formula  which  includes  both  variance  of  demand  and  of  lead  time 

is  derived  by  considering  the  lead  time  demand  as  consisting  of  a  random 

* 

number  of  months  of  random  demand.  This  'lead  time'  includes  both  the  ALT 
and  PLT,  but  we  are  concerned  here  with  the  variance  of  the  PLT. 

_ 

Emanuel  Parzen,  Stochastic  Processes,  Holden-Day,  Inc.,  San  Francisco, 

1962,  p  131.  . . 


CHAPTER  II 


DESCRIPTIVE  STATISTICS  OF  PLT  DISTRIBUTION 

2.1  ARRCOM  Work 

ARRCOM  studied  lead  times  using  the  December  78  NSNMDR.  ALTs  were 

found  to  be  at  least  as  much  of  a  problem  as  PLTs ,  but  only  the  results 

for  PLTs  are  included  in  our  scope.  Most  of  the  ARRCOM  buys  are  for  ASF 

items.  This  study  shows  an  increase  in  the  number  of  buys  as  well  as 

in  the  average  PLT,  at  least  since  July  1976.  The  average  went  from 

* 

4.2  months  in  FY77  to  4.9  months  in  FY78. 

2.2  TARCOM  Work 

TARCOM  studied  a  ten-percent  random  sample  of  the  4000  highest  demand 
items.  Again  ALT  was  found  to  be  at  least  as  variable  and  increasing  as 
fast  as  PLT,  but  we  look  only  at  results  for  PLTs.  The  average  for  buys 
over  $10,000  was  6. 7  months,  and  for  buys  under  $10,000  was  5.3  months. 

For  under  $10,000  PLTs  varied  from  0  to  13  months  with  outliers  up  to  19 
months.  For  over  $10,000  they  varied  from  0  to  15  months  with  outliers  to 
19  months  also.  The  coefficient  of  variation  was  about  0.5. 

2.3  IRQ  Work  with  TSARCOM  Data 


TSARCOM  Troop  Support 

Data  studied  by 

IRO. 

YEAR 

II  SHIPPED 

AVG  PLT 

1969 

1399 

4.9 

1970 

868 

4.4 

1971 

1099 

3.4 

1972 

639 

3.4 

1973 

672 

3.0 

We  observed  a  decreasing  trend  from  1969  to  1973  for  both  number  of 
buys  and  average  PLT  at  TROSCOM. 

For  aviation  items  our  data  base  covers  a  more  recent  period  (following 
table) . 

_  ~  - 

ARRCOM  Memorandum  for  Record,  DRSAR-PES,  February  1979.  A  report  by 
Norman  Trier  is  forthcoming. 

** 

Maryann  Dominiak,  Procurement  Lead  Time  Forecasting  Analysis,  TARCOM 
SA-79-14,  May  1979. 
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TSARCOM  AVIATION  ITEMS 


YEAR 

If  SHIPPED 

AVG  PLT  MOS. 

1972 

13173 

7.0 

1973 

10040 

5.9 

1974 

11517 

6.5 

1975 

12676 

6.9 

1976 

10426 

7.5 

1977 

10813 

7.3 

TABULATED  BY  SHIPMENT  DATE 


There  is  almost  no  observable  trend  even  though  it  was  during  this 
period  that  we  had  severe  economic  problems  and  materials  shortages.  One 
reason  we  do  not  observe  a  trend  is  due  to  the  way  the  data  is  displayed. 
The  averages  are  computed  for  the  year  the  data  became  available  (the 
shipment  date)  rather  than  the  contract  date.  Since  the  trends  would  most 
likely  be  associated  with  the  contract  date,  but  the  randomness  of  PLTs 
causes  contracts  signed  in  the  same  year  to  be  shipped  in  different  years, 
presenting  the  data  by  shipment  date  tends  to  obscure  trends.  However, 
the  alternate  method  of  presentation,  by  contract  date,  tends  to  Introduce 
spurious  trends  due  to  truncation  of  the  PLT  distribution. 


YEAR 

if  SHIPPED 

AVG  PLT  MOS. 

1972 

7741 

6.1 

1973 

13316 

6.7 

1974 

11321 

7.2 

1975 

12852 

6.5 

1976 

9453 

7.1 

1977 

8411 

6.9 

TABULATED  BY  CONTRACT  DATE 

If  we  look  at  this  table  with  the  understanding  that  any  lead  time  over 
one  year  has  been  left  out  of  1977,  over  two  years  from  1976,  etc.,  we  see 
that  there  may  well  be  an  increasing  trend.  From  a  study  of  the  PLT  dis¬ 
tribution,  it  was  found  89%  of  the  PLTs  were  less  than  one  year,  and 
essentially  all  of  the  rest  were  less  than  two  years.  The  mean  lead  time 
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over  one  year  seemed  to  be  about  16.5  months.  Thu9  we  can  estimate  that 
the  average  for  1977  should  be  about  8.3  months  (6.9  x  .85  +  16.5  x  .15), 
which  does  suggest  a  trend. 

The  following  bar  graph  presents  the  empirically  determined  distribution 
of  aviation  component  PLT.  Approximately  21"  (10.5  +  10.6)  of  the  observa¬ 
tions  were  between  three  and  five  months.  To  interpret  the  three  bars  et 
the  upper  end  of  the  distribution,  an  average  of  2.1%  of  the  observations 
appeared  at  each  of  the  six  months  from  13  to  IS,  a  total  of  12.6%  of  all 
the  data.  Note  the  characteristic  departure  from  symmetry,  i.e.  the  heavy  tail 
and  truncation  at  zero  (no  negative  PLTs) .  The  mean  is  about  7.6  months, 
the  mode  about  4.5  and  the  median  a  little  over  6.  Tne  standard  deviation 
of  the  PLT  distribution  should  not  be  measured  over  the  whole  data  base 
because  it  would  be  confounded  with  non-random  changes.  Instead  we  can  get 
an  estimate  of  the  variability  of  the  PLT  distribution,  by  measuring  the 
standard  deviation  over  short  periods  of  time.  You  can  also  get  some  idea 
of  how  much  non-random  trend  there  is  by  comparing  the  standard  deviation 
over  one  year  with  that  over  a  shorter  time,  say  one  quarter.  The  within- 
quarter  and  within-year  standard  deviations  for  the  years  1976  to  1977  are 
.522  and  .537  of  the  means,  respectively;  the  small  increase  is  due  to  the 
lack  of  measured  trend  in  these  years  (possibly  due  to  truncation) . 

The  next  chart  is  an  attempt  to  portray  the  migration  of  PLTs  for  the 
same  aviation  component  from  one  buy  to  the  next.  The  first  PLT  for  an  item 
was  classified  into  one  of  the  six  PLT  classes.  Then  the  next  PLT  for  the 
item  was  classified  using  the  same  set  of  classes.  If  the  second  PLT  fell 
into  the  same  class  as  the  first,  it  was  counted.  Whether  or  not  it  fell 
into  the  same  class,  a  total  count  for  the  class  was  counted  up.  This  was 
repeated  for  each  successive  pair  of  PLTs  for  the  item,  and  done  for  all 
items.  To  interpret  the  chart:  In  13.5%  of  the  pairs  the  first  PLT  fell 
in  the  1  to  75  day  class.  Of  these  pairs  38.2%  of  the  second  buys  also 
fell  in  the  1  to  75  day  class. 
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PLT  CLASS  (MONTHS) 


MIGRATION  OF  PITS 


PLT 

CLASS  (DAYS) 

PERCENT 

PERCENT 

NOT  MIGRATING 

DAYS 

IN  CLASS 

1-75 

13.5 

38.2 

75 

76-150 

22.6 

37.8 

75 

151-200 

15.3 

2A.6 

50 

201-250 

12.0 

20.1 

50 

251-325 

15.0 

27.3 

75 

326- 

21.5 

51. A 

? 

Overall 

35.0 

For  this  particular  set  of  PLT  classes  it  was  found  that  an  average 
of  35%  of  the  buys  were  in  the  same  class  as  the  previous  buy.  If  there 
were  no  correlation  between  successive  PLTs,  this  fraction  would  have 
been  expected  to  be  only  16%. 
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CHAPTER  III 


ATTEMPTS  TO  USE  ANCILLARY  DATA 


3. 1  Materials  of  Which  Item  is  Made 

Variation  between  PLTs  can  be  separated  into  between-buy  for  same 
item  and  between-item  components.  Our  work  with  different  contract  types 
(below)  and  trend  (previous  chapter)  has  suggested  that  each  of  these  com¬ 
ponents  is  of  significant  impact  to  forecasting.  This  chapter  discusses 
attempts  to  find  parameters  which  would  be  useful  in  grouping  items 
to  increase  statistical  confidence  and  effectively  reduce  the  between-item 
component  of  variance.  This  would  be  accomplished  by  dividing  the  universe 
of  items  into  subsets,  called  a  catalog,  so  that  a  homogenous  group  of 
items  could  be  used  to  compute  averages  that  would  aid  in  forecasting  PLT 
for  individual  items  in  the  group. 

Several  attempts  at  breaking  out  items  by  various  parameters  nave  been 
made.  The  following  is  part  of  the  breakout  used  at  M1C0M.  It  shows  how 
PLT  varies  by  type  of  material  used  to  manufacture  missile  components.  It 
is  based  on  survey  of  vendors. 


!>l.T(n.-iy:;) 
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Other  Electronic  Chassis 
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Special  &  Hifth  Reliability 

Electrical /Klecti'onic  Components 
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COMPARISON  OF  ITEM  LEAD  TIME 
WITH  THAT  OF  MATERIALS 


A  TITANIUM 


r 


A  similar  breakout  by  type  of  component  was  obtained  from  Purchasing 
Magazine  based  on  survey  of  purchasing  agents.  This  breakout  also  shows 
change  of  PIT  with  time.  IRO  compared  this  with  the  changes  in  PLTs  for 
aviation  components  and  found  no  obvious  correlation.  (See  graph) 

3.2  Federal  Item  Class  Identifiers 

Navy's  Fleet  Materiel  Support  Office  (FMSO)  defined  a  procedure  [4]  for 
estimating  replacement  factors  based  on  Item  Nomenclature  and  Federal  Stock 
Class  (FSC) .  Another  FMSO  study  [2]  found  that  these  identifiers  were  in¬ 
adequate  for  estimating  unit  price.  Defense  Logistic  Agency's  (DLA)  Electronic 
Materiel  Center  (DESC)  conducted  a  one-time  survey  of  current  and  forecasted  PLT 
(thru  the  National  Association  of  Manufacturer's  Representatives)  by  Item 
Nomenclature.  The  resulting  changes  were  used  to  update  the  PLTs  in  the  re¬ 
quirements  computation  file  and  resulted  in  many  buys  which  brought  up  lagging 
supply  performance  brought  on  by  an  alleged  rise  in  PLTs.  No  attempt  was  made 
to  determine  whether  stock  may  have  been  bought  for  the  wrong  items  as  well  as 
some  of  the  right  items.  Because  DLA  attempts  to  handle  only  fast  moving  items, 
perhaps  an  across-the-board  safety  level  increase  without  the  PLT  change 
excuse  would  have  been  just  as  effective.  Our  conclusion  is  that  there  is 
still  no  evidence  that  Item  Nomenclature  or  FSC  can  be  used  to  classify  items 
to  improve  forecasting  of  PLT. 

3 . 3  Vendor  Estimates 

Bell  Aircraft  Company  maintains  a  file  of  PLTs  for  the  components  they 
make.  AVSCOM  got  a  printed  listing  of  this  file  and  manually  compared  75 
of  them  with  the  properly  adjusted  values  from  the  CCSS  file  and  found  ex¬ 
cellent  agreement.  We  repeated  this  comparison  on  the  entire  file  and  con¬ 
firmed  the  agreement,  but  we  were  uncomfortable  with  this  methodology  because 
the  values  in  CCSS  may  have  been  derived  from  Bell  estimates.  This  is  because 
the  CCSS  values  are  based  on  the  promised  delivery  date  in  the  most  recently 
signed  contract.  We  have  recently  tried  comparing  the  Bell  estimates  with 
the  actual  lead  times  observed  in  the  following  two  years.  The  results  are 
inconclusive  because  of  data  base  limitations,  but  they  are  promising. 
Specifically,  the  errors  were  about  as  large  as  the  worst  found  in  evaluating 
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statistical  forecast  methods  (Chapter  6) ,  but  much  of  this  error  could  be 
explained  as  due  to  bias  which  may  have  resulted  from  limitations  in  the  data. 
Further  study  with  better  data  is  indicated. 

3.4  Unit  Price  as  Stratifier 

Unit  price  should  be  a  good  way  to  group  items  for  PLT  forecasting. 

More  expensive  items  are  likely  to  be  more  complicated  and  thus  take  longer 
to  make.  Raising  capital  may  take  longer,  and  inspection  would  be  more 
thorough. 

CERCOM  found  a  relationship  between  unit  price  and  PLT  for  major  items 
(see  table  below) . 
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CERCOM  RELATIONSHIP  FOR  MAJOR  ITEMS 


PLT  MONTHS 


SUP 

FIRST  BUY 

ADD-ON  BUY’S 

500 

11 

8 

1000 

17 

11 

2000 

19 

13 

3000 

23 

16 

4000 

26 

18 

5000 

27 

19 

6000 

29 

19 

7000 

30 

20 

8000 

32 

21 

9000 

34 

21 

Over  9000 

36 

22 

Source: 

CERCOM  experience 

ending  in  1963. 

for  about 

1000  major  item  contract 

Such  good  results  for  major  items  offers 

hope  for  secondary  items. 

ollowing  distribution  was  developed  for  aviation  secondary  items. 

MONTHS 

SUP 

BUY  COUNT 

AVG  PLT 

STD  DEV 

0-350 

12703 

5.3 

.14 

351-10 

10562 

6.0 

.15 

10.01-22 

10399 

6.4 

.06 

22.01-49 

11527 

6.9 

.11 

49.01-235 

17416 

7.8 

.05 

235.01-3M 

12217 

10.4 

.06 

This  distribution  shows  a  strong  relationship  between  PLT  and  unit  price. 
A  forecast  method  based  on  this  relationship  is  evaluated  below. 
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3.5  Dollar  Value  of  Order 


We  also  found  a  relationship  between  dollar  value  of  buy  and  PLT: 

MONTHS 


$  ORDERED 

BUY  COUNT 

AVG  PLT 

STD  DEV 

0 

95 

.6 

.22 

500 

27064 

5.8 

.08 

2000 

18625 

6.9 

.10 

10000 

18945 

8.3 

.04 

50000 

7328 

10.0 

.07 

100000 

1425 

11.6 

.17 

1000000 

1231 

12.6 

.20 

10000000 

95 

14.0 

.94 

3 

N 

o 

16 

15.5 

1.75 

3 • 6  Contract  Type 

The  variable  which  seemed  to  be  best  related  to  PLT  of  aviation  com¬ 
ponents  Is  the  contract  type  as  used  by  TSARCOM  (formerly  AVSCOM) .  Our  data 
base  contained  buys  coded  with  five  contract  types  as  defined  in  ASPR: 

CODE  DESCRIPTION 

A  Blanket  Purchase  Agreements 

C  Contracts,  including  Letter  Contracts.... 

D  Indefinite  Delivery  Type  Contracts 

F  Delivery  orders .. .other  Government.... 

M  Purchase  Order  -  manual . 

TSARCOM  assigns  a  Basic  Order  Agreement  (BOA)  for  each  aircraft  system. 

Buys  are  simply  calls  on  such  an  agreement.  Most  of  the  buys  seem  to  be 
of  this  type,  called  Blanket  Purchase  Agreement  by  ASPR. 

Our  data  covered  the  years  1972  thru  1977;  in  that  period  there 
were  considerably  fewer  competitive  buys  than  BOA  calls.  Indefinite  de¬ 
livery  type  contracts  and  delivery  orders  on  contracts  of  other  government 
agencies  were  almost  never  used  during  that  period.  Purchase  orders  were 


used  very  frequently  for  buys  under  $10000. 

An  Item  is  usually  bought  using  a  single  type  of  contract  for  a  long 
period;  then  a  new  type  may  be  used.  For  example,  most  aviation  components 
are  bought  using  a  BOA.  Finally  the  item  may  begin  to  be  produced  by 
several  vendors,  and  TSARCOM  will  switch  to  a  competitive  buy.  For  an 
emergency  order  for  a  small  quantity  a  purchase  order  may  be  used. 

The  following  graph  shows  the  quarterly  average  PLT  for  each  of  the 
three  mainly  used  types.  There  is  no  question  of  the  difference  between 
purchase  orders  and  the  other  two  types;  purchase  orders  are  filled  in 
much  less  time. 

The  line  for  BOA  is  noticeably  above  that  for  competitive  contracts 
over  $10000.  This  does  not  mean  that  using  a  BOA  takes  longer  than  the 
competitive  process.  Rather  it  is  an  indication  of  the  type  of  items  that 
are  bought  by  the  two  different  methods.  TSARCOM  uses  the  BOA  for  the  type 
of  aircraft-peculiar  item  that  requires  the  experience  of  firms  like  Bell 
and  Lycoming;  these  items  will  naturally  take  a  long  time  to  produce.  When 
there  is  sufficient  production  experience  there  is  an  attempt  to  buy  on 
open  bid.  By  then  due  to  learning-curve  effects  the  PLT  may  be  considerably 
reduced . 

While  a  slight  increasing  trend  is  evident,  it  does  not  seem  to  be 
different  for  different  contract  types.  Thus  we  have  found  a  good  method 
of  separating  items  (at  least  aviation  items)  of  different  PLT  lengths, 
but  we  have  not  been  able  to  identify  item  types  that  show  different  trends. 
It  would  be  foolhearty  to  believe  that  all  items  really  follow  a  single 
trend,  but  we  have  no  other  evidence  in  our  work  with  aggregate  PLT. 


CHAPTER  IV 


FORECAST  METHODS 

4 . I  Basic  Model 

Our  problem  was  to  find  a  method  of  using  the  available  data,  namely 
the  item's  own  history  as  well  as  averages  by  contract  type,  to  estimate 
the  value  that  was  most  likely  to  be  closest  to  the  PLT  which  subsequently 
occurred . 

Depending  on  which  common  criterion  of  "best  forecast"  we  use,  the 
best  method  would  be  the  simple  average  or  the  median  of  all  the  item's 
history.  The  average  works  best  when  the  criterion  is  root  mean-square 
error  (RMS)  while  the  median  minimizes  the  average  absolute  error  (MAD). 

This  is  true  only  for  a  constant  process  with  random  variation. 

If  this  simple  model  of  the  PLT  process  were  complicated  with  a  fixed 
trend  (like  Inflation?)  or  seasonal  cycles,  they  could  be  identified  and 
removed  before  applying  the  average  or  median.  However,  observation  suggests 
that  this  type  of  process  may  be  described  by  randomly  broken  trends  or  some¬ 
thing  equally  unpredictable.  So  we  want  a  forecast  method  that  can  follow  a 
trend  for  a  while,  but  not  get  locked  into  it  forever.  We  have  had  good  ex¬ 
perience  predicting  demand  using  moving  averages,  so  we  tried  something 
similar  here.  A  moving  average  includes  only  recent  data,  say  the  last  two 
years,  and  thus  does  not  put  any  weight  on  old  trends. 

There  is  a  special  problem  with  PLTs:  The  number  of  observations  in  a 
fixed  time  can  vary.  For  items  that  are  bought  infrequently  there  may  be 
only  one,  or  even  not  any,  buys  in  the  past  two  years.  This  is  where  we  can 
make  use  of  averages  over  similar  items  on  the  assumption  that  they  would 
reflect  recent  economic  conditions. 

We  would  also  like  to  get  a  grip  on  a  recent  trend  by  considering  buys 
that  have  not  been  shipped  by  the  time  the  forecast  is  needed  for  the 
next  buy.  If  we  have  a  contract  that  is  already  a  year  old,  and  the  average 
of  previous  contracts  was  only  nine  months,  it  seems  like  a  good  idea  to 
average  in  the  year  to  get  a  lea'  on  the  trend. 

Finally,  a  note  about  evaluation:  We  felt  that  average  absolute  error 
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was  a  better  criterion  than  RMS  error  because  the  latter  tends  to  give  too 
much  weight  to  large  errors.  However,  most  forecasting  studies  use  RMS 
error.  We  decided  to  rank  the  performance  of  our  various  forecast  methods 
based  on  the  above  ideas  by  average  absolute  error  and  check  this  ranking 
against  that  of  RMS  error.  As  long  as  there  was  little  difference  we  would 
use  average  absolute  error. 

The  following  sections  show  the  formulas  used  for  the  forecast  methods. 


4.2  Notation 


PLT 

N 

F 

DTSGNR 

M 

Note:  1 
CAT  j 


NOW 

R 


the  iC^  PLT  in  the  previous  2  years 

the  last  (most  recent)  PLT 

the  forecasted  PLT 

the  signature  date  of  the  PLT 

the  last  buy  placed  on  contract 
1...N;  J  =  1...M;  M  ^_N 

average  of  the  most  recent  two  quarters  of  the  catalog  average 

* 

PLT  for  a  certain  contract  type;  most  recent  period  ending 
on  the  buy's  PWD  date.  Here  1  £  j  <_  M  +  1  where  the  M  +  1 

buy  is  the  one  we  are  trying  to  forecast, 
the  date  of  the  PWD  of  the  M  +  1  buy. 

N 

1  if  NOW  -  DTSGNR  >_  Z  PLT  /N ;  0  otherwise 
-1  i-1  1 

M 

n  +  r  6. 

J-i  J 


4. 3  Description  of  the  Methods 


PREV  PLT  (current  method  in  CCSS) : 

F  -  PLT. 

2-YR  AVG: 


N 


F 


1 

N 


N 

I  PLT 
i-1 


Note:  The  contract  type  is  assumed  to  be  that  of  the  M  buy  (the  one 
preceding  the  current  buy,  which  is  not  yet  on  contract)  because 
this  is  the  latest  one  that  is  on  contract  and  thus  has  a  known 
type. 
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EXTENDED  AVG: 


E  =  {  E  PLT  +  I  i.  (NOW-DTSGNR. ) } 

R  i=l  1  j=l  j  j 


COMBINATION  M: 


F  -  (cO(F')  +  (1-a)  (CATm+1) 


where  a 


F'  is  either  the  2-YR  AVG  or  the  EXTENDED  AVG  forecast 
and  C  is  a  constant. 


MEDIAN: 


F  *  ~  {PL Ta  +  PLTfi> 


where  A  =  [R/2],  B  =  [R/2  +  1] 


and  the  brackets  mean  "greatest  integer  less  than  or  equal". 


INDEX: 


N  CAT 

F  “ »  L  (^T')  <PLV 


CATALOG: 


F  =  CAT. 
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CHAPTER  V 


EVALUATION  OF  FORECAST  METHODS 


5 . 1  General  Description  of  Methodology 

The  forecast  methods  presented  in  the  previous  chapter  were  evaluated 
bv  feeding  them  real  data  and  comparing  the  resulting  forecast  with  the 
actual  PLT  that  subsequently  occurred.  This  was  accomplished  with  a  com¬ 
puter  program  and  a  data  base  both  specially  developed  for  this  study. 

The  way  it  was  done  may  be  considered  a  deterministic  simulation.  For 
each  procurement  the  program  looked  at  the  buys  in  its  past,  computed  a 
forecast  based  on  the  known  PLTs  of  these  buys  (or  any  other  information 
required  by  the  forecast  method  to  be  evaluated) ,  and  compared  this  fore¬ 
cast  to  the  PLT  of  the  procurement  under  consideration.  The  amount  of 
difference  between  the  forecasted  and  actual  PLTs,  i.e.  the  observed  error, 
was  then  treated  as  a  statistical  sample. 

5. 2  Data  Base 

The  data  base  consisted  of  procurements  for  almost  thirty  thousand 
aviation  components.  Each  procurement  was  characterized  by  the  date  the 
contract  was  signed,  the  type  of  contract,  the  date  by  which  one-third  of 
the  total  order  was  shipped,  a  code  to  identify  the  PLT  as  representative 
or  not  or  based  on  a  promised  delivery  date,  and  the  total  quantity  and 
dollar  value  of  the  order.  The  unit  price  of  the  item  was  derived  by 
dividing  the  last  two  values.  It  was  assumed  that  the  forecast  was  needed 
90  days  before  the  signature  date  of  the  contract.  The  PLT  was  computed 
by  subtracting  the  contract  date  from  the  shipment  date. 

The  figure  shows  the  information  used  in  forecasting  a  single  item. 

This  particular  item  had  eleven  buys.  Characterized  by  shipment  date, 
there  is  one  in  1973,  one  in  1974,  four  in  1975,  two  in  1976,  and  three  in 
1977.  The  contract  type  is  indicated  by  a  letter  at  the  left-hand  end  of 
each  buy.  There  were  six  BOA  calls  in  1973  thru  1975,  four  competitive 
awards  in  1976  and  1977,  and  one  purchase  order  in  1977. 

The  left-hand  end  of  each  line  is  at  the  date  that  a  forecast  was  made. 
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The  tick  in  the  middle  is  the  contract  signature  date,  and  the  right  hand 
end  is  the  shipment  date.  The  PLT  in  days  is  marked  above  the  segment 
representing  the  PLT. 

The  first  buy,  marked  A^,  is  not  considered  an  observation  because 

there  is  no  previous  data  on  which  to  base  a  forecast.  This  particular 

item  has  ten  observations:  A„-A, ,  C,-C.  and  M, .  By  contract  type  it  has 

l  o  1  A  1 

five  BOA  observations,  three  Competitive  contract  observations,  and  no 
Purchase  Order  observations.  The  following  table  gives  the  distribution 
of  BOA  observations;  the  distributions  of  the  other  two  contract  types  were 
similar  to  the  BOA  distribution. 


NUMBER 

OF 

OBSERVATIONS 


PERCENT 

OF  ITEMS  GIVEN 
AT  LEAST  ONE  OBSERVATION 


90. 5% 


8.3  (1.7  each) 


More  than  10 


3 . 3  Editing 

In  general,  an  observation  was  required  to  have  at  least  one  buy 
shipped  in  the  two-year  period  preceding  the  forecast  date  (the  left  hand 
end  of  the  line) .  The  data  base  contained  all  procurements  beginning  or 
ending  in  the  six-year  period  1972  thru  1977,  except  those  whose  PLT  was 
more  than  33  months  or  less  than  one  day  (obvious  errors) ,  which  amounted 
to  less  than  one  percent  of  the  total.  About  65  percent  of  the  buys  were 
not  used  as  observations  because,  for  each,  there  were  no  procurements 
of  the  same  contract  type  shipped  in  the  two-year  base  period  for  that  buy. 

To  eliminate  a  source  of  bias,  only  the  observations  shipped  in  1976 
and  1977  were  used  to  evaluate  forecasts.  Only  for  those  years  could  a 
full  33  month  PLT  observation  be  guaranteed  a  forecast  to  evaluate.  For 
Instance,  if  a  33  month  PLT  was  shipped  at  the  beginning  of  1975,  its 
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two-year  base  period  would  end  at  the  beginning  of  1972;  thus  it  could  not 
be  used  for  evaluating  forecasts  (no  data  before  1972) .  But  a  21  month 
PLT  would  have  a  good  chance  of  having  some  buys  in  its  base  period,  which 
would  end  at  the  beginning  of  1973.  Thus,  if  we  utilized  buys  ending  before 
1976,  we  would  tend  to  over-forecast  thru  no  fault  of  the  forecast  method; 
rather  it  would  be  due  to  faulty  methodology. 

5.4  Definition  of  Aggregate  Statistics 

The  forecast  error  was  computed  by  subtracting  the  actual  PLT  from 
the  forecast,  and  accumulated  using  the  following  formulas: 

MAD  »  AV  ABS  ERROR  =  SUM  ( | ERROR | ) /COUNT 

BIAS  =  Average  Error  =  SUM  (ERROR) /C0UN1 

RMS  =  Root  Mean  Square  Error  =  SUM  (ERROR2) /COUNT 


where  the  SUM  is  taken  over  all  observations  by  contract  type,  and  COUNT 
is  the  number  of  observations  in  the  SUM. 


I 
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CHAPTER  VI 
RESULTS 

6 . 1  Preliminary  Results 

This  chapter  presents  the  results  of  the  study.  First  we  present 
two  secs  of  results  from  other  sources.  TLese  results  are  shown  to  be 
similar  to  ours  for  the  currently  used  forecast  method  (Previous  PLT)  and 
the  TWO-YEAR  AVERAGE  method.  The  remaining  sections  of  the  chapter  present 
our  results  for  all  the  forecast  methods  and  for  demonstration  points  re¬ 
lated  to  forecasting,  all  in  the  format  defined  in  Section  6.2.  Finally  we 
show  the  cost  saving  expected  if  the  best  method  is  implemented. 

The  TARCOM  study  cited  above  also  compared  forecast  methods.  It  was 
found  that  a  simple  two-year  average  outperformed  the  current  policy  (last 
representative  buy) . 

(1977-1978)  TARCOM  RESULTS  (MAY  1979) 


(163  Items) 

NUMBER  OF  OBSERVATIONS  396 
MEAN  PLT  (MONTHS)  5.9 
AVG  ABS  ERROR  FOR  PREV  PLT  2.6 
AVG  ABS  ERROR  FOR  2-YR  AVG  2.5 


Adapted  from  (Procurement  Lead  Time  Forecasting  Analysis,  Maryann 
Dominiak,  TARCOM  SA-79-14,  May  1979).  See  report  for  additional  work  with 
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other  forecast  methods  and  evaluation  criteria. 

The  following  table  shows  the  results  of  IRO  work  comparing 
the  current  forecast  method  with  a  simple  two-year  average.  Again  the 
average  is  favored  slightly  (has  smaller  average  absolute  error) . 

(1966-1973)  TROSCOM  RESULTS  (NOV  1975) 

PROCUREMENT  METHODS 


COMPETITIVE 

DIRECT  PURCHASE 

SOLE  SOURCE 

ALL 

NUMBER  OF  OBSERVATIONS 

5818 

3127 

265 

9210 

MEAN  PLT  (DAYS) (MONTHS) 

4.7 

4.4 

4.5 

4.6 

AVG  ABS  ERROR  FOR  PREV  PLT 

2.5 

2.2 

1.8 

2.4 

AVG  ABS  ERROR  FOR  2-YR  AVG 

2.4 

2.1 

1.6 

2.3 

*Recent  work  at  ARRCOM  also  confirmed  the  conclusion  that  the  PREV  PLT  method 
does  not  work  well. 
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The  next  table  gives  results  in  the  same  format  for  IRO  work  with 
TSARCOM  aviation  components. 

(1972-1977)  AVSCOM  RESULTS  (JULY  1979) 

CONTRACT  TYPE 


BOA 

COMPETITIVE 

PURCHASE-ORD 

ALL 

NUMBER  OF  OBSERVATIONS 

5120 

1052 

3784 

10009 

MEAN  PLT  (MONTHS) 

8.75 

6.5 

4.6 

6.9 

AVG  ABS  ERROR  FOR  PREV 
PLT 

3.5 

2.8 

2.2 

2.9 

AVG  ABS  ERROR  FOR 

2-YR  AVG 

3.4 

2.7 

2.1 

2.8 

Contains  53  buys  on  indefinite  delivery  contracts  and  delivery  orders. 

The  above  results  with  TARCOM,  TSARCOM  Troop  Support  and  TSARCOM 
Aviation  items  all  reflect  the  same  conclusion:  Average  absolute  error 
is  smaller  when  recent  PLTs  are  averaged  than  when  only  the  most  recently 
observed  PLT  is  used  as  the  forecast. 

6.2  Final  Results  with  Aviation  Data 

■In  the  following  sections  we  present  the  complete  results  of  our  work 
with  the  TSARCOM  Aviation  items.  The  presentation  format  is  as  follows: 

Objective 

Ranking  by  Average  Absolute  Error  (MAD)  and  Root-Mean-Square 
Error  (RMS) 

Spread  of  Forecast  Error  (where  examined) 

Discussion 

The  results  are  presented  by  contract  type,  and  for  all  types  pooled.  Under 
each  Is  given  the  number  of  months  average  absolute  error  followed  by  its 
rank  in  parentheses,  followed  by  the  RMS  and  its  rank.  Notice  the  high 
degree  of  consistency  across  contract  type  and  between  MAD  and  RMS.  No 
BIAS  figures  are  shown  because  in  every  case  it  was  very  small. 
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RANKING  OF  FORECAST  METHODS 


Objective:  To  find  the  best  PLT  forecast  method. 


CONTRACT  TYPE 


Forecast  Method 

BOA 

COMPETITIVE 

PURCHASE  ORD 

All 

PREV  PLT 

3.5(8) 

4.7(10) 

2.8(8) 

3.9(10) 

2.2(9) 

3.1(9) 

2.9(8) 

4.1(9) 

2-YR  AVG 

UNSHIPPED  INCL 

3.4(6) 

4.5(6) 

2.7(6) 

3.8(7) 

2. 1(6. 5) 

3. 0(6. 5) 

2.8(6) 

3.9(6) 

MEDIAN 

3.4(5) 

4.5(5) 

2. 7(4. 5)3. 7(5) 

2. 1(6. 5) 

3. 0(6. 5) 

2.8(5) 

3.9(5) 

AVERAGE 

COMBINATIONS 

3.3(4) 

4.5(4) 

2. 7(4. 5)3. 7(5) 

2.1(5) 

2.9(5) 

2.8(4) 

3.9(4) 

C“1 

3.1(1) 

4.0(1) 

2.4(1) 

2.4(1) 

3. 2(2. 5) 

1.8(2. 5) 

2.5(1) 

3.5(2) 

C=2 

3.1(2) 

4.0(2) 

2.4(2) 

3.2(1) 

1.8(1) 

2.5(1) 

2.6(2) 

3.4(1) 

C=3 

3.2(3) 

4.1(3) 

2.5(3) 

3.3(3) 

1.8(2. 5) 

2. 5(2. 5) 

2.6(3) 

3.5(3) 

INDEX 

3.4(7) 

4.6(7) 

2.7(7) 

3.8(8) 

2.1(8) 

3.0(8) 

2.8(7) 

4.0(8) 

CATALOG  UP 

3.5(9) 

4.6(8) 

3.0(10) 

3.8(9) 

3.0(10) 

3.6(10) 

3.3(10) 

4.1(10) 

CATALOG  CONTR 
TYPE 

3.7(10) 

4.7(9) 

2.9(9) 

3.6(4) 

2.0(4) 

2.7(4) 

3.0(9) 

3.9(7) 

SPREAD  OF  FORECAST  ERROR  (Months  95th  Percentile) 

BOA  COMPETITIVE  PURCHASE  ORD 

PREV  PLT  9  76 

COMB  M-l  8  65 

Discussion:  There  is  a  13%  reduction  of  avg  absolute  error  between  the  best 
method  (COMB  C*l)  and  the  current  method  (PREV  PLT) .  The  improvement  is 
consistent  across  contract  type  and  for  root-mean  square  and  spread  of  errors. 
The  poor  performance  of  the  CATALOG  method  based  on  the  UP  vs  PLT  relation¬ 
ship  may  be  explained  by  the  small  difference  in  PLT  between  unit  price 
classes  (except  the  highest).  See  table  in  Section  3.4  at  bottom  of  page  17. 
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FIRM  DELIVERY  DATES 


Objective:  To  estimate  Improvement  due  to  using  firm  delivery  dates 
in  forecast. 


CURRENT  METHOD 


BOA 


COMPETITIVE  PURCHASE  ORD 


ALL 


WITHOUT  FIRM 
WITH  FIRM 


3.5(2)  4.7(2) 

3.2(1)  4.4(1) 


2.8(2)  3.9(2) 

2.7(1)  3.8(1) 


2.2(2)  3.1(2) 

2.1(1)  3.0(1) 


2.9(2)  4.1(2) 
2.7(1)  3.8(1) 


TWO-YEAR  AVG 


BOA 


COMPETITIVE  PURCHASE  ORD 


ALL 


WITH  PRES.  AGE  3.3(2) 
WITH  FIRM  3.2(1) 


4.5(2)  2.7(2)  3.7(2) 

4.3(1)  2.7(1)  3.7(1) 


2.1(1)  2.9(1) 

2.1(2)  3.0(2) 


2.8(2)  3.9(2) 
2.7(1)  3.8(1) 


Discussion: 

There  were  very  few  firm  delivery  dates  in  the  data  base.  This  ex¬ 
periment  was  accomplished  by  using  the  actual  PLTs  of  the  base-period  buys 
even  when  they  had  not  been  shipped  in  time  for  the  forecast.  Thus  the 
improvement  must  be  considered  an  upper  bound  on  that  achievable  by  using 
firm  delivery  dates  in  this  way. 

Using  the  firm  delivery  date  of  the  most  recently  placed  contract  in¬ 
stead  of  the  actual  PLT  of  the  latest  shipped  buy  achieved  a  5.5%  reduction 
in  average  absolute  error.  This  was  consistent  over  contract  type  and  for 
RMS  error. 

Replacing  present  age  with  firm  delivery  date  in  2-YR  AVG  improved 
forecasting  a  little,  but  not  for  purchase  orders. 
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CONTRACT  TYPE 


Objective:  To  see  if  there  is  any  advantage  to  ignoring  contract  type 
to  increase  the  number  of  observations  in  the  two-year  average. 

TWO-YEAR  AVG  INCLUDING  UNSHIPPED 

BOA  COMPETITIVE  PURCHASE  ORD  ALL 

SINGLE  CONTR.  TYPE  3.3(2)  4.5(2)  2.7(1)  3.7(1)  2. 1(1. 5)  2.9(2)  2.8(1)  3.9(2) 
IGNORE  CONTR.  TYPE  3.1(1)  4.4(1)  2.8(2)  3.8(2)  2. 1(1. 5)  2.9(1)  2.8(2)  3.9(1) 

Discussion: 

The  advantage  of  additional  observations  is  cancelled  by  the  lack  of 
uniformity  of  the  multi-contract  data.  The  slight  improvement  was  not 
consistent  over  contract  type  or  between  avg.  absolute  error  and  RMS  error. 
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RECENT  BUY  EXPERIENCE  AND  PLT  LENGTH 

In  order  to  see  whether  PLTs  were  shorter  when  there  were  recent  buys 
average  PLTs  were  computed  for  the  two-year  period  1976  thru  1977. 

Following  table  compares  average  PLT  for  those  buys  with  another  buy 
in  the  previous  two  years  vs  those  without.  The  averages  are  given  by 
contract  type,  and  for  all  contract  types  together  (including  a  few  buys 
on  indefinite  delivery  contracts  and  a  few  delivery  orders  not  included 
in  the  type  breakout) . 

CONTRACT  TYPE 

BOA  COMPETITIVE  PURCHASE  ORD  ALL 

NEW  BUYS  9.4(5603)  9.3(519)  5.0(3154)  7.9(9262) 

RE-BUYS  8.8(5120)  6.5(1052)  4.6(3784)  6.9(10009) 

Overall  It  takes  an  average  of  one  month  less  to  produce  items  with 
recent  experience. 


PLT  FORECAST  VARIANCE 


Objective:  To  estimate  the  variance  of  the  forecast  distribution. 

Following  table  gives  the  coefficient  of  variation  and  mean  PLT  for 
three  contract  types  and  a  pooled  total.  Data  covers  years  1976  thru  1977. 
Results  are  given  for  best  forecast  method  and  for  current  method  for 
comparison.  Based  on  RMS. 

COEFFICIENT  OF  VARIATION  BY  CONTRACT  TYPE 


BOA 

COEFFICIENT  OF  VARIATION 

COMPETITIVE 

PURCHASE  ORD 

POOLED 

COMB  C-l 

.461 

.495 

.558 

.499 

PREV  PLT 

.535 

.597 

.683 

.587 

MEAN  PLT  (MONTHS) 

8.8 

6.5 

4.6 

6.9 

NUMBER  OBS. 

5120 

1052 

3784 

9956 

Discussion: 

The  forecast  error  variance  for  the  best  forecast  method  is  about 
15Z  less  than  that  for  the  current  method. 
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6.1  Cost  Savin 


The  following  charts  can  be  used  to  estimate  the  cost  savings  achievable 
with  the  proposed  forecast  method.  The  first  chart  is  for  stock  fund  (ASF) 
and  the  second  for  procurement  appropriation  (PAA)  of  aviation  components. 

Both  charts  assume  that  the  safety  level  calculation  includes  the  variance 
of  the  procurement  lead  time  forecast. 

For  each  component  the  budget  strat  data  was  used  to  compute  the  safety 
level  and  order  quantity.  From  these  the  availability  was  projected.  Each 
x-axis  value  is  simply  the  sum  of  the  dollar  values  of  the  safety  levels 
of  all  the  components;  the  y-axis  value  is  the  demand-rate  weighted  average 
of  the  availabilities.  Four  different  values  of  backorder  penalty  cost  were 
used  to  trace  the  curves.  The  upper  curve  is  based  on  a  coefficient  of  vari¬ 
ation  of  PLT  forecast  equal  to  .5,  the  value  for  the  Combination  method  with 
C-l;  and  the  Lower  curve  is  based  on  .6,  the  value  for  the  Current  forecast 
method. 

To  estimate  typical  savings  look  at  the  ASF  graph.  At  an  availability 
of  0.885  on  the  vertical  scale,  the  two  curves  are  about  $15  million  apart 
as  measured  on  the  horizontal  scale.  This  means  we  would  expect  a  $15  million 
saving  in  safety  level  by  using  the  Combination  forecast  method  as  opposed 
to  the  current  method. 


CHAPTER  VII 


CONCLUSIONS 

From  the  work  done  by  TARCOM  and  ARRCOM  and  our  own  work  with  AVSCOM 
data  we  conclude  that  the  attempt  to  follow  trends  by  basing  the  PLT  fore¬ 
cast  on  only  a  single,  last  representative  buy  has  failed.  An  average 
of  the  last  two  years  of  representative  buys  performs  better  as  a  forecast 
method.  This  confirms  the  high  degree  of  variability  of  PLT  values  observed 
by  TARCOM,  ARRCOM  and  1R0  studies.  There  is  usable  information  in  the  pro¬ 
curement  history  of  the  item,  and  an  average  of  the  recent  buy  experience 
is  a  better  way  to  access  this  information  than  just  the  most  recent  buy. 

We  have  also  found  that  the  information  inherent  in  the  fact  that  a 
recent  order  has  reached  an  age  greater  than  the  average  of  Drevious  orders, 
can  be  harnessed  to  improve  the  forecast.  Presumably,  these  unshipped  orders 
give  us  the  handle  on  trend  that  we  were  trying  to  get  by  using  only  the 
last  representative  buy.  By  this  approach  we  are  using  even  more  recent 
data,  buys  that  are  not  yet  "representative." 

Another  finding  is  that  a  two-quarter  average  of  all  buys  for  the  same 
contract  type  that  is  to  be  used  for  the  buy  whose  PLT  we  are  attempting 
to  forecast,  can  be  used  to  add  valuable  information  when  the  item's  own 
history  is  insufficient.  The  fact  that  using  this  catalog  average  by  it¬ 
self  as  a  PLT  forecast  method  gave  poor  results,  but  using  it  to  fill  in 
for  items  with  few  recent  buys  gave  the  largest  improvement  of  any  of  the 
techniques  tried,  suggests  that  its  value  is  mainly  as  a  supplement.  This 
is  also  confirmed  by  the  poor  performance  of  the  INDEX  method. 

Since  the  catalog  average  did  perform  better  than  the  currently  used 
method,  it  may  also  have  some  value  as  a  forecast  method  by  itself  when 
there  is  no  recent  history,  i.e.,  no  shipments  in  the  last  two  years.  In 
this  application  it  may  tend  to  underforecast  slightly  because  we  found  a 
slight  difference  in  the  average  PLTs  for  items  with  no  recent  buys  and 
those  with  recent  buys.  Items  with  no  shipments  in  the  preceding  two  years 
had  PLTs  about  one  month  longer. 

An  experiment  suggested  that  including  PLTs  calculated  from  the  firm 
delivery  dates  of  unshipped  contracts  in  the  two-year  average  instead  of  the 
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present  age  Improved  performance  a  little.  However,  more  work  needs  to 
be  done  on  this  using  more  suitable  data.  If  results  are  positive,  perhaps 
current  vendor  estimates  could  also  be  used  in  forecasting  PLT. 

Attempting  to  group  the  catalog  by  a  unit  price  stratification  instead 
of  contract  type  deteriorated  forecast  performance.  This  was  surprising 
because  the  stratification  seemed  to  separate  different  values  of  PLT 
efficiently.  This  work  could  be  extended  by  applying  an  inflation  adjust¬ 
ment  to  the  unit  prices. 

Even  though  our  best  forecast  method  represents  a  significant  improve¬ 
ment  there  still  remains  an  average  absolute  error  of  two  and  one-half 
months.  We  have  no  way  of  knowing  whether  this  error  represents  an  irredu¬ 
cible  random  variation  or  a  predicatable  change  given  other  information. 
Further  empirical  work  with  Box-Jenkins  auto  and  cross-correlation  should 
reveal  something  along  these  lines.  A  contact  at  the  University  of  Wisconsin 
has  expressed  interest  in  this  work.  Experimenting  with  Markov  or  Kalman 
models  may  also  provide  insight. 

One  of  the  goals  of  this  study  was  to  determine  whether  a  significant 
increase  in  requirements  may  cause  an  increase  in  PLT.  Work  was  begun  with 
a  stratification  (suggested  by  CERCOM)  on  dollar  value  of  the  order,  but 
an  unfortunate  accident  with  the  data  base  terminated  the  work  before  any 
results  were  obtained. 

Other  problems  that  could  be  studied  include  the  question  of  independence: 

Is  variability  independent  of  stock  position,  or  will  we  see  expe¬ 
dited  shipments  when  assets  are  low? 

Theoretical  questions  on  variance  estimation  can  be  studied  empirically: 

Can  we  assume  that  orders  cross  (an  earlier  order  delivered  later)? 

What  approach  do  we  use  to  measure  variance  when  the  PLT  proocess 
is  nonstationary? 

Should  we  be  accounting  for  the  variance  of  PLT,  and  especially  ALT,  at 
all?  The  effect  of  implementing  our  PLT  variance  as  a  coefficient  of  varia¬ 
tion  will  have  the  effect  of  assuming  the  same  amount  of  variation  for  the 
ALT  as  the  PLT.  Should  they  be  different? 


APPENDIX  I 


LACK  OF  AGREEMENT  BETWEEN  CCSS  FILES 

In  developing  the  data  ba9e,  some  problems  arose  in  verifying  the 
accuracy  of  the  data.  The  PLT  averages  showed  a  lack  of  consistency  between 
different  files.  The  following  notes  give  our  findings  based  on  a  manual 
check  of  fifty  important  items.  The  files  are:  National  Stock  Number  Master 
Data  Record  (NSNMDR)  and  MILSCAP  Master  File  (MMF) . 

NSNMDR  Sector  10  seems  to  agree  with  MMF. 

PROLT  in  Header  (NSNMDR)  seems  too  big  compared  to  Sector  10;  however, 
results  with  a  small  group  of  items  show  reverse  effect. 

Sector  13  seems  to  agree  with  Header  (except  for  rounding) . 

Freeze  code  usage  was  unexplained  in  two  follow-up  calls.  This  may 
be  cause  of  disagreement  between  Header  and  Sector  10. 

The  following  graph  compares  our  data  base  (MMF  and  Sector  10)  with 
the  Budget  Strat  (Sector  13) .  The  two  lines  show  the  dollar  weighted  average 
PLTs  from  Sector  13  for  procurement  appropriation  and  stock  fund  items  respec¬ 
tively.  The  lower  lines  are  an  average  weighted  by  number  of  buys  and  a  PLT 
based  on  the  last  buy  of  the  year,  both  from  our  data  base  (MMF  and  sector  10) . 
It  was  thought  that  the  line  based  on  the  last  buy  of  the  year  would  match  the 
sector  13  data  because  the  latter  is  presumably  based  on  the  latest  represen¬ 
tative  buy.  It  can  be  seen  that  this  line  follows  the  stock  fund  line  closely 
until  the  end  of  1972.  At  that  point  the  two  lines  diverge.  Freeze  codes  were 
implemented  in  1975,  and  could  thus  explain  some  of  the  difference  from  that 
year  on.  We  have  no  explanation  for  the  divergence  in  the  year  1973  and 
the  three-month  difference  in  1974.  Dollar  weighting  of  the  sector  13  re¬ 
sults  could  be  emphasizing  longer  PLTs ,  but  we  know  by  checking  individual 
items  that  differences  occur  that  can  not  be  explained  by  relative  weighting 
of  different  items. 
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APPENDIX  II 


DATA  PROCESSING  +  EDITING 

The  following  chart  elaborates  the  data  processing  that  was  performed 
in  developing  and  analyzing  the  data  base.  Following  the  chart  are  des¬ 
criptions  of  the  programs  referenced  on  the  chart.  Finally  there  is  a  table 
showing  the  number  of  procurements  edited  from  the  data  base,  and  the  order 
in  which  which  it  was  done. 
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OTHER 

SUMMARIES 


MEASURES 


DESCRIPTION  OF  PROGRAMS 


MMFEX  collects  contract  data,  CLIN  data  and  shipment  data  for  each  contract 
If  1/3  of  total  of  all  CLINs  for  any  NSN  ordered  on  the  contract  haa 
been  shipped,  a  record  is  output  to  the  data  base. 

Each  record  contains  NSN,  DT-CONTRACT,  Date  by  which  1/3  shipped,  Total 
amount  shipped  (qty  and  $) ,  Contract  PUN,  etc. 

REDUC  selects  Sector  10  from  extract. 

COMBX  reformats  Sector  10  data  to  look  like  data  base  record.  Keeps  only 
representative  buys. 

ELMDP  eliminates  duplicate  records  (due  to  overlap  of  MMF  and  Sector  10 
coverage) . 

FASE1  averages  PLTs  by  contract  type  and  quarter. 

PLTSM  simulates  forecast  methods  and  collects  and  displays  error  statistics 
for  evaluation. 


EDITING  SUMMARY 


74824  Buys  In  original  data  base. 

205  Removed  because  item  had  no  NSN. 

573  Removed  because  PLT  was  less  than  one  day  or  more  than  1000  days. 
1299  Removed  because  shipped  before  1972. 

45029  Removed  because  there  were  no  shipments  in  two-year  period 
ending  90  days  before  contract  was  signed. 

17709  Removed  because  the  shipment  was  not  in  1976  or  1977. 

46  Removed  because  they  were  indefinite  delivery  contracts. 

7  Removed  because  they  were  delivery  orders  on  contracts  of  other 
Government  agencies. 

9956  Observations  remain. 

Contract  Type  Breakout: 

5120  BOA 

3784  Purchase  Orders 
1052  Competitive  Contracts 
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help.  Using  entire  nomenclature  did  not  help.  No  method  of  grouping 
gave  acceptable  results. 

[3]  Hill,  Joseph  S.,  "Analysis  of  Production  Lead  Time  for  Missile  Repair 
Parts  with  Cable  Assemblies  and  Wiring  Harnesses,"  April  1975,  USAMC 
Intern  Training  Center,  LD33435A. 

Results  suggest  that  PLT  is  influenced  by  total  contract  cost,  but 
data  was  insufficient. 

[4]  ALRAND  Working  Memo  231  (ASO) ,  "Family  Replacement  Factors  for  Aviation 
Materiel,”  3  September  1974. 
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